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CAPACITY  OF  THE  STATIONARY  GAUSSIAN  CHANNEL 


Abstract 

Information  capacity  of  the  stationary  Gaussian  channel  is  determined 
under  the  assumption  that  both  the  channel  noise  and  the  constraint  are 
defined  by  rational  spectral  densities.  The  results  complement  a  well-known 
result  due  to  Holsinger  and  Gal lager. 

Introduction 

The  information  capacity  per  unit  time  of  the  stationary  Gaussian  channel 
is  a  problem  having  solutions  that  date  back  roughly  30  years.  Fano’s  1961 
book  [1]  contains  a  solution  based  on  his  ingenious  "waterfilling"  approach. 
However.  Fano’s  approach  was  somewhat  heuristic,  and  incomplete.  Although  he 
discussed  the  necessity  to  impose  frequency  constraints  on  the  transmitted 
signal,  appropriate  constraints  were  not  incorporated  into  his  analysis.  Thus, 
his  result  does  not  give  a  solution  for  physically  realizable  noise  processes 
(e.g.,  rational  spectral  density). 

A  careful  solution  to  the  capacity  problem  is  given  in  Gal lager’s  book 
[2],  with  the  basic  approach  attributed  to  Holsinger.  In  that  work,  the 
message  process  is  passed  through  a  linear  filter  before  being  transmitted: 
the  filter’s  properties  are  such  that  the  transmitted  signal  is  constrained 
both  in  its  total  energy  and  the  distribution  of  the  energy  over  frequency. 

The  Hoi singer-Gal lager  solution  can  be  formally  obtained  from  Fano’s  result  by 
replacing  the  noise  spectral  density  (in  Fano’s  result)  with  the  function 

Sjj/lHl,  where  H  is  the  Fourier  transform  of  the  filter. 

However,  the  Holsinger-Gal lager  solution  does  not  completely  solve  the 
capacity  problem.  In  this  paper  we  complete  the  solution.  It  will  be  seen 
(under  mild  assumptions)  that  the  new  results  to  be  given  here  are  complemen¬ 
tary  to  those  of  Holsinger-Gal lager  (H-G);  that  these  results  and  the  H-G 
result  taken  together  exhaust  all  possible  situations,  including  infinite 
capacity;  and  that  the  new  results  enable  one  to  transmit  signals  that  occupy 
a  larger  portion  of  the  effective  noise  bandwidth.  This  can  lead  to  a  larger 
value  of  capacity  (for  fixed  signal  total  energy)  than  can  be  obtained  under 
the  H-G  assumptions,  provided  that  there  is  sufficient  freedom  in  the  selec¬ 
tion  of  the  constraint  on  the  signal. 


Problem  Formulation 

The  channel  is  Y  =  X  +  N,  where  N  represents  stationary  Gaussian  noise,  X 
is  the  transmitted  signal  process  (independent  of  N) ,  and  Y  is  the  channel 
output.  It  will  be  assumed  that  N  has  a  rational  spectral  density,  $^.  The 

T  2 

constraint  to  be  placed  on  the  signal  process  is  given  by  EIIX  11^  i  PT,  where 

T 

P  >  0  is  fixed,  T  >  0,  X  is  the  signal  process  X  restricted  to  the  parameter 
set  [0,T],  and  II •  11^  j  denotes  a  reproducing  kernel  Hilbert  space  (RKHS)  norm 

with  parameter  set  [0,T].  11*11^  will  be  defined  by  a  covariance  function 


1 


corresponding  to  a  rational  spectral  density  function  will  denote  the 

RKHS  of  r^  with  parameter  set  [0,»);  will  be  the  RKHS  of  r^  with 


parameter  set  [0,T].  The  quantity  sought  is  C^(P),  defined  by 

C^(P)  =  lim  i-C^(PT),  where  C^(PT)  is  the  supremum  of  I[X^,Y^]  over  all  signal 
T-*» 

T  T  2 

processes  X  that  have  sample  paths  a.s.  in  ^  and  satisfy  EIIX  11^  ^  i  PT. 

T 

The  capacity  C^(PT)  for  T  >  0  is  determined  in  [3]. 

To  realize  the  constraint,  one  may  proceed  as  follows.  Let  h  be  the  L 2 

Fourier  transform  of  Let  M  be  the  message  process.  Set 

EjjM^Odt  *  g-  (constraint  on  M). 
and  define  X^  and  X1  by 

XT(t)  5  jJh(t-s)M(s)ds  t  €  [0,T] 

X*(t)  s  J*Jh(t-s)M(s)ds  t  *  0. 

Since  the  integral  operator  in  L2[0.T]  with  h  as  kernel  is  strictly  positive, 
I[XT,YT]  =  I[M.YT]. 

,XT,2  ^ 

This  gives  EIIXTllJ  t  $  Elixjlj  =  E  J^,  — (X)dX,  where  x  is  the  L2 

w 

Fourier  transform  of  x.  Thus, 

|XT 

1  T9  1  TO  1  09  I  (OB  I 

f EIIX  "w,T  *  t  El,x«"w  =  r  ES-~ 

W 

2 

=  f  EC  =  f  E£j*(x>  |2dX  (since  |h|2  =  *w) 

w 

=  Y  E  2ir/^ajM2(t)dt  =  Y  2TrE/jM2(t)dt  £  P. 

1  T  2 

Moreover,  lim  sr  EIIX  T  exists  and  equals  P. 

Th»  1  w,t 


The  basic  approach  here  is  to  compute  by  using  the  results  on 

and  letting  T  This  procedure  is  very  complicated;  the  results  given  in 

T 

[3]  for  C^(PT)  are  stated  in  terms  of  the  spectrum  of  a  self-adjoint  operator 


cJ(PT) 
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4-  4. 

Spin  L2[0,T],  satisfying  Rjj  T  =  T(IT+ST)R^  j.  Here,  and  T  denote 

the  integral  operators  in  L2[0,T]  whose  kernels  are  the  covariance  function  of 

T 

N  and  r^.  respectively.  C^(PT)  then  depends  on  0^,  the  smallest  limit  point  of 
the  spectrum  of  S^,;  the  eigenvalues  (if  any)  of  that  are  strictly  less  than 

T 

0-pi  and  the  value  of  PT.  The  expression  for  C^(PT)  takes  several  forms, 
depending  on  these  quantities. 

We  also  identify  two  other  capacities.  Let  &^(P)  be  the  set  of  all  X 
(defined  on  [0,“))  such  that  for  every  T  >  0,  XT  €  H^  T  with  probability  one, 
-  1  T  2 

and  lim  EIIX  11^  £  P.  Next,  let  if  =  Sf„( P)  be  the  set  of  all  wide  sense 

T~*°  ’ 

(w.s.)  stationary  processes  X  with  a  SDF  (spectral  density  function),  denoted 

$ 

_Co  *v 

by  such  that  is  bounded  and  (X)dA  £  2ttP.  The  corresponding 

capacities  are  then 

C^P.a)  =  sup(Ti^  ^  I(XT.YT):  X  €  ^(P)}; 

T-*» 

C^P .if)  =  sup(T7^  i  I(XT,YT):  X  €  <yP)}. 

Th® 


Lemma  1.  0>W(P)  C  9^(P). 

Applying  Lemma  1,  it  is  clear  that  (^(P;*)  £  C^P ;9) .  The  differences 


between  the  classes  of  signal  processes  leading  to  Cy(P),  C^(P ;&),  and  P ;if) 


—  -  IT  T 

can  be  summarized  as  follows.  (^(P)  =  lim  f  Cy(PT) .  where  C^PT)  is  computed 

Thoo 

(separately  for  each  T)  by  maximizing  I(XT,YT)  over  all  admissible  XT.  Thus, 
for  example,  if  cJ(PT)  =  I(XT,YT)  and  cJ’tPT')  =  I(XT',YT).  for  T’  >  T.  then 

T  T' 

it  is  not  necessary  that  X  (t)  =  X  (t)  for  t  in  [0.T],  However,  in  computing 

—  T  T 

C^(P;&),  the  signal  process  X  is  fixed;  one  computes  I(X  ,Y  )  for  this  fixed 


For 


process,  all  T  >  0;  and  C^(P,#)  is  the  supremum  over  all  X  €  &  of 
-  ITT  — 

lim  j  I(X  , Y  ).  This  also  applies  to  C^(P ;£/•),  with  the  further  restriction 
T-*» 

that  the  signal  process  is  w.s.  stationary.  However,  it  is  not  necessary  that 


?.n 


>n/  _ _ 

Ity  Codes 
*  nnd/or 


□  □ 


X  in  3^  satisfy  EIIX^II^  £  PT  for  any  finite  T. 

Let  1  +  0  =  lim  $m(X)/$„,(X)  .  with  0  =  »  if  the  limit  does  not  exist; 

|xh»N  ff 

-1  <[  0  £  ®.  0  is  an  important  parameter  for  the  capacity  problem. 


Main  Results 


Theorem  1.  C^P)  =  Cy(P;$)  =  C^fP :if). 

This  result  has  the  following  implications  for  determining  the  capacity 
C^(P).  First,  it  is  sufficient  to  consider  only  all  signal  processes 


X  s  (X(t).  t  *  0: 


XT  €  1^  T  a.s.  for  all  T  >  0.  and  lim  i  EIIXTII^  T  $  P}  rather 

T-w* 


T  T 

than  all  families  of  signal  processes  {X  :  T  >  0}  =  {X  (t) ,  t  €  [O.T]: 

X^  €  a.s.  and  EIIX^ll^  j  £  PT,  all  T  >  0}.  Second,  these  processes  can  be 

taken  as  w.s.  stationary. 


We  now  give  the  value  of  C^(P).  The  results  will  be  separated  according 
to  the  value  of  the  parameter  0. 


—  T 

Theorem  2.  If  0  =  -1,  then  C^( P)  =  00 .  C^(PT)  =  00  can  be  achieved  for  every 
T  >  0.  using  a  signal  process  with  finite-dimensional  support. 


Theorem  3.  If  -1  <  0  <  «.  let  A  =  {X:  *N(X)  <  (1 +0)$w(x)>- 


‘Wv 


i.  if  p  *  2?  s\[l  +  9 '  then 

w 


Vp>  *  5T  J'A1°g[(l+«)^(M]-B>  *  5^  *  jaL#<x) 


1 


fex)  -  (He)] 


dX. 


II.  If  ♦<>-**00 

VP>  =  4?  log[(H9)A(P)^(X)]dA 


dX ,  then 


where  Aj  =  {X:  $jy(X)  £  (1+0)A(P)$^(X)}  and  A(P)  £  1  is  uniquely  defined  by 

p  =  2K1[<i+e>A<p> 

In  Part  I,  C^(P)  *  P/[2(l+0)].  In  Part  II,  (^(P)  £  P/[2(1+0)A(P)]  2 
P/2(l+0) . 
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Theorem  4  fHolsineer- Gal  lager  I .  If  0  =  »,  then 

vp>  =  ^  v°g[B<P)*iHdx' 

where  Ag  =  {X:  $N(X)  i  B(P)$W(X)}  and  B(P)  is  uniquely  determined  by 


Theorem  5.  In  Part  II  of  Theorem  3  and  in  Theorem  4,  capacity  can  be  attained 
with  a  stationary  Gaussian  signal  process  X,  having  spectral  density  $x>  For 

Part  II  of  Theorem  3.  $x  is  defined  by 

$X(X)  =  (1+0)A(P)  -  [f^/lw](A)  X  in  A1 

=  0  otherwise. 

For  Theorem  4,  $x  is  defined  by 

*x(\)  =  B(P)  -  [V$W](X)  X  ln  A2 

=  0  otherwise. 

In  considering  Theorems  2-5,  it  can  be  seen  that  Part  II  of  Theorem  3  is 
quite  similar  to  Theorem  4;  in  fact.  Theorem  4  can  be  obtained  by  substituting 
B(P)  =  (1+0)A(P)  in  Part  II  of  Theorem  3.  This  results  from  the  fact  that  as 

0  -»  *,  A(P)  must  converge  to  zero  (fix  and  let  vary).  However,  Part  I  of 

Theorem  3  is  quite  different  from  Theorem  4.  To  gain  insight  into  this,  one 
can  compare  these  results  with  those  contained  in  Theorem  3  and  Corollary  4  of 
[3],  Specifically.  Part  I  of  Theorem  3  above  should  be  compared  with  Part  (a) 
of  Theorem  3  in  [3];  Part  II  of  Theorem  3  above  should  be  compared  with  Part 
(b)  of  Theorem  3  in  [3J;  and  Theorem  4  should  be  compared  with  Corollary  4  in 
[3]. 

The  question  of  attaining  capacity  (in  particular,  by  a  stationary 

T 

process)  In  Part  I  of  Theorem  3  is  still  open.  For  any  finite  T,  C^(PT)  cannot 

be  attained  when  the  conditions  of  Part  I  are  satisfied  (see  Theorem  3(e)  of 

[3]). 


To  illustrate  the  difference  in  the  effect  of  the  constraints  on  the 

2  2  2 

properties  of  the  transmitted  signal,  suppose  that  $^(X)  =  a  /(X  +a  ).  Then 

0  =  »  (Theorem  4  constraint)  requires  that  $^(X)  =  P^(X)/Pg(X)  have  numerator 
polynomial  P^  of  degree  at  least  4  less  than  the  degree  of  the  denominator 
polynomial  P^.  However,  the  model  of  Theorem  3  permits  to  be  given  by  P^/P^ 
such  that  the  degree  of  P^  is  2  less  than  that  of  P^.  This  means,  in  particu¬ 
lar.  that  the  sample  pjaths  of  the  transmitted  signal  for  the  constraint 
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satisfying  Theorem  4  must  be  twice-differentiable,  with  Lg  derivative.  The 

sample  paths  of  the  transmitted  signal  obeying  a  constraint  satisfying 

Theorem  3  need  only  be  once-differentiable  with  Lg  derivative.  Thus,  the 

transmitted  signals  using  the  constraint  of  Theorem  3  need  not  be  as  smooth  as 
those  obeying  a  constraint  satisfying  Theorem  4,  and  consequently  can  utilize 
more  of  the  frequency  spectrum. 

Conclusions 

The  new  results  given  here  show  that 

(1)  Attention  can  be  restricted  to  w.s.  stationary  signals  (Theorem  1); 

(2)  The  new  results  for  the  value  of  the  capacity  (Theorem  2  and  Theorem  3) 
are  complementary  (under  the  assumptions  used  here)  to  the  result  of  Holsinger 
and  Gal  lager;  together,  these  results  exhaust  all  possible  values  of  the 
capacity; 

(3)  'The  constraints  permitted  in  obtaining  Theorem  3  enable  the  signal  pro¬ 

cess  to  use  more  of  the  available  noise  bandwidth  than  signal  processes 
obeying  the  constraints  of  the  Holsinger-Gal lager  model;  if  there  is  freedom 
to  choose  the  constraint,  then  the  capacity  can  be  increased  by  using  a  con¬ 
straint  as  in  Theorem  3.  ,  / 

All  of  the  above  results  are  obtained  under  a  basic  assumption:  both  the 
noise  and  the  constraint  are  defined  by  rational  spectral  densities.  This 
restriction  is  imposed  in  order  to  obtain  results  that  can  be  directly 
compared  with  the  result  of  Holsinger  and  Gal lager.  More  general  results  will 
be  given  in  [4]. 
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